1. Field of the Invention
The present invention relates to an augmented reality system.
2. Description of the Prior Art
Augmented reality refers to the process of capturing a video image of an environment, and augmenting that video image with computer graphic elements (typically depicting virtual objects such as game characters that are to appear as if part of the environment). To provide a convincing augmentation, it is desirable for these computer graphic elements to be displayed in a manner consistent with the environment, particularly in terms of scale, range of motion, and viewing angle. Thus for example if the video image of the environment (referred to herein also as the ‘scene’) depicts a floor or table-top from a particular view point, it is desirable that the computer graphic elements are scaled and oriented so as to appear to properly rest on that surface, and to move on it in a physically consistent fashion.
However, the determination of the scale and orientation of elements of the scene, and hence of the virtual objects that are superposed on them, is a non-trivial task.
Referring now to FIG. 1, consequently in a typical augmented reality application a so-called fiduciary marker 1010 of a known size is included in the environment to be captured by the video camera, in order to provide an easily detectable and known feature whose scale and orientation can be more readily estimated.
Such fiduciary markers are typically high-contrast (e.g. black and white) so providing scope for pattern recognition in even relatively poor image capture conditions (e.g. with low resolution image capture, poor lighting, and the like). The patterns themselves usually comprise a thick quadrilateral boundary containing an asymmetric distribution of blocks or tiles, which enable a disambiguation of the orientation of the marker. Consequently the recognition process typically provides an estimation of the position (x and y axis position in the image) and rotation (by virtue of the marker's asymmetry) of the fiduciary marker within the captured video image.
Optionally in addition the distance of the fiduciary marker (its z-axis position) from the video camera may be estimated by comparing the size of the fiduciary marker in the captured image with a known size of the fiduciary marker (e.g. its size in pixels at a distance of 1 meter); in other words, its scale.
Similarly, optionally a tilt of the fiduciary marker (i.e. its deviation from the x-y plane in the z direction) may be estimated by comparing the aspect ratio of the fiduciary marker in the captured image with the known aspect ratio of the fiduciary marker, and/or using other techniques such as detecting foreshortening (where a rectangular shape appears trapezoidal) or other distortions of the fiduciary marker in the captured video image.
The generated graphical overlay typically comprises a virtual graphics element that can then be superposed over the fiduciary marker, with the estimated position, rotation distance and tilt of the fiduciary marker used to modify the virtual graphics element as applicable.
The augmented image is then output to a display.
The subjective effect of this process is that the output video image comprises a graphical element (for example a monster or a castle) in place of the fiduciary marker, typically replicating the estimated position, rotation, distance and tilt of the fiduciary marker and hence also the surface of the environment upon which the marker has been placed.
However, it will be appreciated that in such augmented reality systems the user's suspension of disbelief, which makes them complicit in the illusion that the graphical overlay is part of the captured scene, is at least in part dependent upon the reliability with which the graphical overlay matches or interacts with some real-world aspect of the scene. This reliability is frequently dependent upon estimates of the position and orientation of the fiduciary marker within the scene.
Thus it is desirable to reduce the occasions upon which such estimates cannot be reliably made, and to improve the consistency of estimates that are made.